The present invention relates generally to the field of telecommunication, more particularly to the field of telecommunications in which graphical user icons are used for communication in which behavior is learned.
This application is a continuation-in-part of co-pending application Ser. No. 09/415,769 filed Oct. 8, 1999, assigned to the assignee of the present application and which is hereby incorporated by reference.
In visual representation-based environments in which a user controls a virtual representation to interact with other users, the visual representation personality can be initially selected by the user to reflect generally the user""s own behavior and personality. For example, as described in co-pending application Ser. No. 09/415,769 (hereinafter xe2x80x9cparent applicationxe2x80x9d) a user can select from a variety of personality types to represent the user, including xe2x80x98hip-hop,xe2x80x99 xe2x80x98upper-class,xe2x80x99 xe2x80x98rockerxe2x80x99, or the like. The embodiments described in the parent application also allowed a user to further specify the behavior of his or her visual representation by setting mood intensity values. The mood intensity values modify the behavior of the visual representation (or visual representation) to allow the visual representation to appear, on a sliding scale, positive, indifferent, or aggressive. However, in these embodiments, the user defines the visual representation""s behavior explicitly through personality and mood selections and explicit gesture commands. Because the settings are generically defined, there is a limitation on how the personalities can be tailored to match the personality of the individual user. Additionally, to modify the visual representation""s behavior, the user adjusts the different settings that control the visual representation""s behavior. This interface provides a limitation on the virtual representation experience, as the user is constantly aware that the user is explicitly in control of the visual representation""s behavior.
Accordingly, a new system is needed in which the visual representation""s behavior is modified through the natural course of using the visual representation, without requiring explicit directives provided by the user. Additionally, a system is needed in which the visual representation""s behavior is modified to match more closely the behavior of the user.
A system, method, and apparatus are disclosed in which utterances entered by a user are processed to identify patterns of behavioral movements executed by the user""s visual representation. Once identified, the patterns are used to generate behavioral movements responsive to new utterances received from the user, without requiring the user to explicitly alter the behavioral characteristics selected by the user. Thus, the behavior of the user""s visual representation is modified without requiring explicit user directives, and the modified behavior is generated responsive to the user actual communication activity, thus ensuring that the visual representation""s behavior is more closely related to the user""s own personality.
More specifically, in one embodiment an application module parses an utterance generated by a user to determine the presence of gesture commands. An utterance is an input text string and optional gesture commands entered by a user through a keyboard or voice commands and processed by the application module of the present invention to generate visual representation behavioral movements. Gesture commands are explicit commands made by a user to have the visual representation act out or animate a behavioral movement or series of movements to convey a particular idea or emotion. For example, a gesture command could have the visual representation bow, wave, skip around, raise an eyebrow, or the like. The user issues a gesture command by providing a gesture command identifier along with the text portion of the utterance. Thus, an exemplary utterance is: xe2x80x98Hello (wave) how are you?xe2x80x99, where xe2x80x98(wave)xe2x80x99 is a gesture command provided by the user to have the user""s visual representation animate a hand wave. In a preferred embodiment, each utterance is parsed to determine the presence of gesture commands. If a gesture command is found in an utterance, the utterance is stored for behavioral learning processing.
A stored utterance is analyzed in context of the existing stored utterances to determine if the stored utterances provide the basis for creating a new behavioral rule. A behavioral rule generates behavioral movements based on natural language processing, as described in the parent application. A behavioral rule comprises a context, a weighting, and a behavioral movement. A context is a gesticulatory trigger and optionally associated word or words. A gesticulatory trigger is a linguistic category designed to organize word types into common groupings. A weighting defines the probability that the specific behavioral movement will be used; in one embodiment, the scale is 1 to 10, where 10 is a 100% probability. An exemplary behavioral rule is: {10 * Negative* shake head}. Thus, the weight value 10 specifies that the behavioral movement xe2x80x98shake headxe2x80x99 will be performed when a word classified as a Negative gesticulatory trigger is identified in an utterance.
In one embodiment, newly stored utterances are analyzed to generate different contexts associated with the behavioral movement, using the Specific gesticulatory trigger for the contexts. Gesticulatory triggers can include classes such as Prepositions, which includes any preposition, Referents, which includes words that refer to other objects such as xe2x80x98thisxe2x80x99 or xe2x80x98that,xe2x80x99 Negatives, which includes negative words such as xe2x80x98noxe2x80x99 or xe2x80x98not,xe2x80x99 or Specifics, which are simply any specific word designated by quotation marks, such as xe2x80x98Hixe2x80x99 or xe2x80x98Helloxe2x80x99. In one embodiment, pre-assigned words are provided with each personality type in a lexicon. Each word in the lexicon has an associated behavioral movement and/or a gesticulatory trigger class. Thus, when a specific word such as xe2x80x98noxe2x80x99 is encountered in an utterance, the lexicon is checked to determine whether the word xe2x80x98noxe2x80x99 is present in the lexicon, and if it is present, to which behavioral movement it is associated, and/or to which gesticulatory trigger class it belongs.
For example, for the utterance xe2x80x98Hello (wave) how are you?xe2x80x99, generated contexts using the Specific gesticulatory trigger include xe2x80x98Hello (wave)xe2x80x99, xe2x80x98Hello (wave) howxe2x80x99, xe2x80x98Hello (wave) how arexe2x80x99, xe2x80x98Hello (wave) how are youxe2x80x99, xe2x80x98(wave) howxe2x80x99, (wave) how arexe2x80x99, etc. The different words are different Specific gesticulatory triggers. To determine if any of the contexts should be used as a basis for a new behavioral rule in an embodiment in which contexts are stored, the contexts of the existing utterances in the log are compared with the new contexts. If a context appears in the log at a frequency above a threshold, then the context is used as the basis for a new behavioral rule. For example, if the threshold is 5 appearances, and the context xe2x80x98Hello (wave)xe2x80x99 appears for the fifth time in the new logged utterance, then a new behavioral rule based on xe2x80x98Hello (wave)xe2x80x99 is generated.
In one embodiment, the new behavioral rule is generated by using the Specific gesticulatory trigger as the gesticulatory trigger for the behavioral rule, assigning an arbitrary weighting, and applying the associated gesture as the behavioral movement for the rule. Thus, in the xe2x80x98Hello (wave)xe2x80x99 example, the new behavioral rule is defined as {10, xe2x80x98Helloxe2x80x99, wave} the arbitrary weighting of 10 ensuring that a wave is animated each time the word xe2x80x98helloxe2x80x99 is recognized in an utterance transmitted by the user. However, in a preferred embodiment, the new behavioral rule is then used to modify existing rules, or create more generally applicable rules. Accordingly, the application module of the present invention examines current rules to determine if more general rules can be created from the existing rules. For example, if a new rule is created as {10, xe2x80x98yupxe2x80x99, nod}, and xe2x80x98yupxe2x80x99 is classified as an-Affirmative, the application module examines existing rules to determine whether other specifics using words belonging to the Affirmative gesticulatory trigger class and having the same associated behavioral movement are defined in existing rules. For example, if a rule exists defining {10, xe2x80x98yesxe2x80x99, nod}, and both xe2x80x98yupxe2x80x99 and xe2x80x98yesxe2x80x99 are in the lexicon and classified as being Affirmatives, then a general rule {10, * Affirmative*, nod} is created, thereby generalizing from two specific rules to a more general behavioral rule. Thus, a new, more general rule is created based on the user behavior, which provides the ability for the visual representation to learn. For example, if the user types in a new Affirmative, such as xe2x80x98yeahxe2x80x99, the application module will check the lexicon to determine a gesticulatory trigger corresponding to xe2x80x98yeahxe2x80x99. If xe2x80x98yeahxe2x80x99 is listed as being an Affirmative, the more general rule {10, * Affirmative*, nod} will cause the visual representation to nod, in addition to whatever behavioral movement is already assigned to xe2x80x98yeahxe2x80x99, even though the user had never before issued a gesture command requesting that the visual representation nod its head after receiving the word xe2x80x98yeahxe2x80x99. This new behavior is xe2x80x98learned.xe2x80x99
In one embodiment, new general rules are not created unless the number of existing rules that could create a behavioral rule exceeds a threshold. This embodiment provides control over how persistent a user""s behavior must be before a new rule is created. Thus, in accordance with the present invention, new behavior is implicitly provided in accordance with the present invention, and the new behavior is generated responsive to user actions.